The sense of tragedy is that the world is not a pleasant little nest made for our protection, but a vast and largely hostile environment, in which we can achieve great things only by defying the gods; and that this defiance inevitably brings its own punishment. It is a dangerous world, in which there is no security, save the somewhat negative one of humility and restrained ambitions. It is a world in which there is a condign punishment, not only for him who sins in conscious arrogance, but for him whose sole crime is ignorance of the world around him.
The Human Use of Human Beings: Cybernetics and Society
Norbert Wiener
The last 400 years have seen social changes unlike any earlier period. As our technological reach has grown, our machines threaten to become more than our tools. Our creations seem to slip out of our control and become our masters.
In the early 1950’s Norbert Wiener wrote a slim book, The Human Use of Human Beings: Cybernetics and Society, that was a set of meditations on the relationship of man to his machines. Weiner, who died in 1964, was a American mathematician and philosopher who coined the term “Cybernetics” to describe the complex, unpredictable behavior of simple control systems. In the 50’s, the concern was with automata, robots capable of independent action like those controlling factories or missile systems. It’s striking how different the attitude was toward calculating machines, what we would broadly call “computers”. At the time, these were still large, simple tools without any autonomy. One fed them punch cards and they provided printouts. Weiner clearly saw the changes that telecommunication would bring to automation, worrying that these intelligent machines would move further out of the control of man, talking to each other.
But Wiener was more afraid of man in organizations than organizations of machines:
I have spoken of machines, but not only of machines having brains of brass and thews of iron. When human atoms are knit into an organization in which they are used, not in their full right as responsible human beings, but as cogs and levers and rods, it matters little that their raw material is flesh and blood. What is used as an element in a machine, is in fact an element in the machine. Whether we entrust our decisions to machines of metal, or to those machines of flesh and blood which are bureaus and vast laboratories and armies and corporations, we shall never receive the right answers to our questions unless we ask the right questions.
The Monkeys Paw of skin and bone is quite as deadly as anything cast out of steel and iron. The djinnee which is a unifying figure of speech for a whole corporation is just as fearsome as if it were a glorified conjuring trick.
The hour is very late, and the choice of good and evil knocks at our door, I find it coming back seated on the whirlwind.
Weiner was one of the early systems thinkers, a group that realized that when individual components were put into an interacting system, the behavior of the whole could not be predicted from the behavior of the individual parts. As The Human Use of Human Beings: Cybernetics and Society makes clear, this is true when the parts are logical systems in a computer, mechanical feedback loops in a robot, neurons in a brain, or people organized into a corporation or political party. Weiner and those who followed destroyed hopes and fears of easy prediction of a clockwork, Newtonian universe. The Large World is a place where simple models don’t hold.
Eventually, these fears abated as we avoided nuclear catastrophe and totalitarianism seemed to recede into history. As Wiener predicted, this was an illusion as we are once again threatened by human machines of death and new “bureaus and vast laboratories” armed Big Data and machine learning algorithms. I believe we are entering another stage in our need to reassert our human ability to make our own decisions and do our own work, resisting the urge to surrender to our own creations.
